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Abstract. The evolution of games as an alternative to traditional simulations in the military context has been 
gathering momentum over the past five years, even though the exploration of their use in the serious sense has 
been ongoing since the mid-nineties. Much of the focus has been on the aesthetics of the visuals provided by 
the core game engine as well as the artistry provided by talented development teams to produce not only 
breathtaking artwork, but highly immersive game play. Consideration of game technology is now so much a part 
of the modeling and simulation landscape that it is becoming difficult to distinguish traditional simulation solutions 
from game-based approaches. But games have yet to provide the much needed interactive free play that has 
been the domain of semi-autonomous forces (SAF). The component-based middleware architecture that game 
engines provide promises a great deal in terms of options for the integration of agent solutions to support the 
development of non-player characters that engage the human player without the deterministic nature of scripted 
behaviors. However, there are a number of hard-learned lessons on the modeling and simulation side of the 
equation that game developers have yet to learn, such as: correlation of heterogeneous systems, scalability of 
both terrain and numbers of non-player entities, and the bi-directional nature of simulation to game interaction 
provided by Distributed Interactive Simulation (DIS) and High Level Architecture (HLA). 


1. INTRODUCTION 

The worlds of traditional simulation and serious 
games are converging in many ways. Each of these 
worlds is seeking traction in the military training 
market and each discipline brings its own strengths to 
the challenge of military training. Traditional 
simulation has always focused on high fidelity, 
academic accreditation, and validation - building 
systems that had lofty goals, but often unwieldy 
solutions. These ambitious solutions often attempted 
higher fidelity and more overall simulation capability 
than was necessary to provide baseline training 
value. By contrast, game development has been 
consumer driven and deadline focused with less 
emphasis on fidelity, demonstrating good results with 
relatively short development cycles. Despite the 
game industry’s success, there is still lingering doubt 
when it comes to connecting lower-fidelity game play 
to formal training objectives. The US Department of 
Defense (DOD) is keen to find ways to merge the 
approaches to provide the best training value. We 
feel the hybrid approach should focus on “rightsizing” 
the amount of simulation and fidelity for each 
application. Additionally we believe the future of 
military training must incorporate Non-Player 
Characters (NPCs) to derive maximum training value 
and satisfaction from participants. NPC development 
has long been viewed as “too hard” - but by applying 
the same “task worthy” approach, we should be able 
to provide the right amount of character automation 
for each application. 


2. GAMES AS SIMULATIONS 

2.1 Are Games and Simulations the Same 
Thing? 

Key distinctions between simulations and military 
games can be made in terms of origin and 
expectations. The need for modern constructive 
military simulations grew from the need to replace live 
field exercises and tabletop war games with a 
solution that was both realistic and cost-effective. As 
these exercises moved to electronic environments, 
trainers envisioned multi-part systems that could not 
only implement a war-game, but also to allow 
analysis, trainee performance measurement, and 
After Action Review (AAR) capabilities. Validation 
and Verification (V&V) of these simulations was 
considered a vitally important and often problematic 
part of the system development. Initial simulations 
required very powerful mainframes or mini- 

computers. As microcomputers evolved and became 
ubiquitous and cheap, simulation customers saw the 
advantages of moving to that platform. 

The growth of micro computing was also a huge 
economic boon to the gaming industry - everyone 
had a PC and many found free time to play. Games 
based on military or near-military themes were 
naturally engaging - they involved danger, weapons 
use, and strategy that appealed to the prime early 
demographic - young men. High-fidelity 

representation of true military tactics was not 


283 


necessary, and although some games were fairly 
realistic, there was never any formal V&V process. 
To satisfy the requirement that a game must be 
entertaining and immersive, much of the development 
effort went into creating realistic 3D worlds with 
animated characters and vehicles whose appearance 
could be tailored to game-player preferences. 

In the mid 1990s, the DOD embarked on an 
ambitious program to create a joint simulation 
environment called Joint Simulation System (JSIMS) 
(FY01 Annual Report Joint Simulation System, 2001). 
JSIMS boasted a complex, distributed architecture 
(The JSIMS Program and Architecture, 1997) aimed 
at providing a single simulation solution for all the 
services (Tiron, 2003). Unfortunately, delays, cost 
overruns, internal disagreements, and system 
complexity spelled doom for JSIMS and the program 
was cancelled. An Analysis of Alternatives (AoA), 
conducted by the DOD detailed goals and methods 
for shaping the future of joint training (Gardner & 
Hartman, 2004). The AoA study spurred the adoption 
of technologies that were not traditionally considered 
part of the modeling and simulation community. 
Specifically, the AoA recommended a hybrid 
approach to reinvent the way training systems are 
developed for the DOD that included incorporating 
gaming solutions and innovative acquisition 
techniques. Already worldwide simulation customers 
had become increasingly interested in using game 
approaches. They recognized early on that the great 
size of the game industry, predicted to approach $50 
billion by 2011 (Szalai, 2007), was effectively driving 
innovation. 

As the traditional simulation industry begins to adopt 
some game industry practices, it is important to note 
that, while all games may be simulations, not all 
simulations are games. Games may not attempt to 
realistically represent the emulated environment or 
tactics - but they could still be considered “low-fidelity 
simulations.” Simulations, however, are usually not 
developed with the specific goal of reality escape or 
performance reward (e.g. moving to a new level, 
scoring points) as games are. It is possible that 
reward systems might be useful for training, in 
coming together to produce the next generation of 
simulations, we must view requirements with a fresh 
perspective and derive a hybrid product that best fits 
the training needs of the DOD rather than 
preconceived notions of what constitutes simulation 
versus games. Forward thinking companies have 
begun to merge with other companies to satisfy the 
need, e.g. Kynogon and Autodesk (Autodesk 
Acquires Kynogon SA, 2008) and BioTech and 
Presagis (Engenuity Technologies Acquires BGT 
BioGraphic Technologies, 2005), while other 
companies independently develop technologies that 
span both industries (CityScape 1.7: Real Cities Real 
Fast, 2009). 

2.2 How much simulation do you need? 

Simulations, both traditional and game-based, vary 
greatly in the fidelity of their representation of entities 
and the environment in which they exist. This is 
usually related to the original intent of the simulation. 


A simulation developed for large scale military 
exercises will usually support large terrain areas at a 
low to mid level of fidelity (10+ meter) whereas one 
built based on a “first person shooter” (FPS) will 
usually have a relatively small terrain area at very 
high resolution (sub 1 meter) and include building 
interiors. 

Other areas where fidelity or implementation differs 
among simulations include: 

• Aggregation level - are simulation objects entities 
or units (e.g. a soldier versus a platoon)? 

• Kinematic, sensor and combat models - are they 
fully physics-based including flout of projectiles 
and weather effects on sensors or are they 
effects-based with a “die roll” followed by a 
lookup in a probability of hit/probability of kill (p- 
h/p-k) or detection tables? 

• Level of 3D modeling and animation of entities - 
are individual entities observable in a 3D 
environment or is a symbolic representation in 
2D enough? 

• Number of units - are large groups large groups 
such as populations modeled statistically or 
explicitly as individual entities? 

• Automation of simulation objects - are high-level 
orders available or do objects require significant 
low-level management of behaviors? 

The primary challenge is selecting the right simulation 
components for the task at hand. This is complicated 
by the difficulty of managing user perceptions of their 
simulation needs. For example, a user that has seen 
a high fidelity flight simulator may be resistant to 
using a low to mid fidelity flight model, even though 
the only way to observe the modeled aircraft in the 
target system is as a symbol on a C2 display. 

2.3 Interoperability 

The simulation community has long understood the 
need to develop standards which promote 
interoperability between simulation components. 
These standards range from communications 
protocols such as DIS, HLA and TENA 1 to file and 
message protocols (e.g. MSDL 2 , C-BML 3 ) and 
beyond. 

While no formal standards have been developed, the 
game community has converged on pseudo- 
standards owing to the nature of the components 
they have needed to develop. Comparable 
components developed by different companies have 
evolved similar APIs because they perform 
analogous actions and they are designed to be used 
within the same game development frameworks. 

There are two main areas of interoperability that 
simulations have had to deal with that many game 
systems have not: terrain correlation and sharing of 
simulation objects. It is crucial that all participants in a 
distributed simulation provide position data that is 
correlated to an agreed-upon datum. For many 


1 Test and Training Enabling Architecture 

2 Military Scenario Definition Language - SISO-STD-007-2008 

3 Coalition - Battle Management Language - SISO (under 

development) 

284 



game-based systems the notion of exporting data to 
another simulation (game-based or traditional) is 
completely foreign. Even games which are meant to 
be used by large numbers of players in different 
locations are based on a homogeneous set of servers 
and clients with known capabilities. Adapting a 
game-based simulation to accept externally 
controlled simulation objects can present significant 
challenges (Scolaro, McNamara & Little, 2008). 

2.4 Success of Traditional SAF Approaches to 
Agent Development 

Traditional SAF and Computer Generated Forces 
(CGF) approaches to Human Behavior 
Representation (HBR) have had some success in 
supporting the needs of DOD customers. Pew and 
Mavor (1998) summarized 18 months of study by the 
Panel on Modeling Human Behavior and Command 
Decision Making. The panel focused on realism in 
HBR based on psychological, organizational, and 
sociological theory and generated recommendations 
based on what they observed. They focused on the 
most ambitious, high-fidelity HBRs, ones that 
attempted to represent full human cognition, and 
were used to develop agents tailored to very specific 
military areas. Citing the extreme difficulty of the 
tasks of HBR and cognitive modeling the panel 
recommended the collection and dissemination of 
human performance data to aid in model 
development, creation of accreditation procedures for 
agents, demonstrated agent validation, and accepted 
analysis capabilities. The study documented the 
wide variety of approaches used to create CGFs and 
the lack of coordination and consensus in the 
modeling community. Also, the modeling and 
simulation community may have become too insular 
in its approach to software development. Many SAF 
solutions have been developed as Government Off- 
The-Shelf (GOTS) products or demonstrations with 
no real follow on market. They have been very costly 
to produce and have only a small customer base. 
Traditional HBR solutions may have also been too 
ambitious, trying to represent all aspects of human 
behavior, and cognition in particular, in all products. 

Some companies have achieved wider commercial 
success by selling partial HBR solutions to a greater 
number of customers. For example pathfinders, 
applications that calculate a travel route through a 
navigation mesh, are used in many commercial 
games (Yap, 2002). Many companies including 
Presagis, Kynogon, and Xaitment now have path 
finding products. These path finding systems were 
not aimed at emulating higher level cognitive 
behavior; rather they were developed to solve game 
design and play problems by allowing artists and 
software developers to work together to improve the 
speed to market of a title by creating compelling 
interactive scenes for the consumer market. Game- 
based pathfinders had the additional burden of 
needing to support a wide range of game engines 
which, in turn, supported a production pipeline of 
commercial tools from a variety of manufacturers. 
The result was development of efficient, effective, 
adaptable middleware. Traditional HBR solutions 
had never achieved that level of modularity but 
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customers were beginning to demand it. This shift in 
thinking opened the door for other game based 
middleware approaches in the areas of physics, 
scene-graphs, artificial intelligence and interface 
design. 

One of the earliest examples of the use of a path 
engine to solve military research problems was the 
development of the Crowd Federate at the Virginia 
Modeling and Simulation Center (VMASC) in 2003 
(Flanagan, 2008). Since then, there have been 
numerous examples of the use of path engines to 
animate complex scenes where many NPCs would 
be needed to represent a population. One example 
is the army’s use of Al-lmplant to automate large 
crowds at the Institute for Creative Technologies 
(Lawlor, 2007). The use of commercial based path 
planning engines has mostly been limited to 
automation of large numbers of characters in scenes 
where traditional SAF-based systems would have 
had difficulty navigating and managing the scale of 
animation. However, traditional SAF approaches do 
have the advantage of a richer behavior 
representation allowing more complex interaction. It 
seems clear that many applications could benefit 
from a combination of the two approaches - or better 
yet a new approach borrowing the best from both. 

3. THE FUTURE: PERSISTENT, SOCIAL 
ENVIRONMENTS 

3.1 Persistent Social Communities 

Virtual worlds are emerging as the next platform for 
both games and traditional simulations. The platform 
already boasts a market sector with revenues 
approaching $2 billion a year (Economic Activity in 
Virtual Worlds, 2006). Beginning as extensions of 
Massively Multiplayer Online (MMO) games, virtual 
worlds have become environments that elude 
categorization. Both business and pleasure activities 
are conducted in virtual worlds, which are really seen 
as 3D collaborative spaces for social networking of all 
types. They have evolved from the traditional 
escapism of games to virtual extensions of the real 
world. Virtual Worlds Review (2006), discusses the 
many current applications of virtual worlds including: 
commercial gaming (e.g. World of Warcraft, Habbo 
Hotel), socializing (e.g. Second Life), education (e.g. 
Mokitown), political expression (AgoraXchange), and 
military training (Forterra Systems - On-Line 
Interactive Virtual Environment). 

Figure 1 shows how both traditional simulations and 
games are converging on virtual worlds as the 
platform of choice. Moving to virtual worlds has many 
implications for the future of simulation. As the figure 
shows, even as virtual spaces evolve into training 
and analysis venues the social aspect remains. 
Players in virtual worlds expect social interaction no 
matter what the focus of the virtual space. These 
spaces are also persistent because they are hosted 
on the internet and available 24/7. The combined 
effects of increased social expectations and 
persistence must spur new technology development 
from both the gaming and simulation communities. 
For example, a classic problem with commercial 
virtual worlds is that, if there is not active communal 



participation, use dwindles quickly. The success of 
these worlds relies on the active participation of 
human role players driving avatars to perform 
functions key to an objective. 




Figure 1: Evolution from Traditional Simulation and 
Gaming to Virtual Worlds 


3.2 NPCs as a Solution 

The growth of virtual worlds for simulation and 
gaming has introduced a new requirement to provide 
non-player-characters (NPCs) that can move, 
communicate, and act in predictable and useful ways. 
Figure 2 shows an NPC assisting a human 
represented by an avatar in Second Life. The site is 
a Second Life location, or slurl, for the company 
MASAGroup (Empowering Life, 2009). The purpose 
of the NPCs is to show the company’s products and 
services to visiting human-controlled avatars. 



Figure 2: NPC Communicates with Human in 
Second Life 


There are four key areas where NPCs could support 

humans in virtual worlds: 

1. Presence: arrival into a virtual location can feel 
very similar to arriving in a real but unfamiliar 
location. The use of NPCs can assist in 
reassuring a player or visitor that they are in the 
right place and are supported. 

2. Performance Support: NPCs also provide the 

equivalent of online help by having specific 
knowledge valuable to a player or visitor in a new 
location. This knowledge includes a thorough 
understanding of the venue and all of its features 
to assist in orienting the play to the space and 
their task. 

3. Vigilance: virtual spaces focus a great deal of 
attention into a very narrow visual channel. NPCs 
can support the human by making them aware of 
events and ensuring that if they are not paying 


attention due to fatigue or distraction that they 
can be brought back to task subtly. 

4. Role-Playing: as serious gaming and military 

training use virtual spaces, on-demand NPC 
teammates and adversaries will grow in 
importance 

4. THE IMPORTANCE OF PLAUSIBLE HUMAN- 
LIKE BEHAVIOR 

4.1 HBR in the New World 

While nothing matches human adaptability and 
flexibility, NPCs will need to supplement human role- 
players to enhance training in virtual worlds and 
beyond. For serious game applications NPCs can 
help to direct and focus activity to meet training 
objectives. They can even act as intelligent tutors 
providing real-time feedback and coaching. 

The challenge for HBR is to find a practical middle 
ground between the high fidelity, high cost of many 
traditional HBR approaches and the relatively 
narrowed scope of the path finding that often 
constitutes game Al. This middle ground must not 
only provide a useful and compelling level of 
functionality for NPCs but also be easy to integrate 
within systems along the traditional simulation, game, 
virtual world spectrum. Additionally, HBRs should be 
ready to integrate with other HBRs in order to 
maximize the strengths of each. 

4.2 Approaches to HBR 

During the 1990s alliances between industry, 
government, and academia worked to create 
integrated cognitive architectures to be used to build 
CGFs in simulations such as Modular Semi- 
Automated Forces (ModSAF), and Joint Semi- 
Automated Forces (JSAF). The Agent-Based 
Modeling and Behavior Representation (AMBR) effort 
compared many of the key behavior modeling 
technologies (Gluck & Pew, 2005). Included in the 
comparison were: Elements of ACT-R, Soar, and 

EPIC (EASE), Distributed Cognition (DCOG), 
Cognition as a Network of Tasks (COGNET), and 
Atomic Components of Thought - Rational (ACT-R). 
Although the AMBR evaluation team did not 
specifically rank the architectures, it compared 
cognitive agent performance on multitasking and 
category learning to human performance using a 
common simulation test-bed. The research 
concluded that all of the approaches had merits and 
weaknesses, and that no single approach emerged 
as a clearly superior HBR representation, but that the 
state of the art was strong from a theoretical 
perspective. 

While the architectures developed in the 1990s focus 
almost exclusively on cognition, the following decade 
saw growth in representation of behavioral factors 
traditionally considered to be outside cognition, such 
as culture and emotion. The rise of these factors 
acknowledges that behavior is more than a 
consequence of pure logic. Decision making 
encompasses “fuzzier” areas such as personal 
preference, affects, desires, and belief systems 
(Evertsz, Ritter, Busetta, and Bittner, 2008). 
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Additionally, socio-cultural factors, such as leader 
and follower behavior, can have a significant effect on 
behavior (Silverman, Bharathy, Nye, & Eidelson, 
2007). As the field matures the relative influence of 
all factors on behavior will evolve. NPC architectures 
must be able to respond quickly to new advances in 
HBR. 

Al for games has followed a somewhat different path 
than traditional HBR for simulations. First, the term 
Al has been used to represent any “intelligent” or 
“automatic” behavior on the part of NPCs. Game Al 
can be hardcoded reactions to simple stimuli, 
mathematical algorithms to deal with specific 
problems like path finding, or a more complex 
framework including behavior and knowledge 
representation (Isla & Gorniak, 2009). Game Al 
generally has a more pragmatic goal than traditional 
HBR - commercial viability. Development lifecycles 
must be constrained and products must be robust 
and reliable. Performance and attractiveness are 
critical as game users expect to be immersed in an 
entertainment experience that runs on their home 
computer. Fidelity of behavior representation will be 
sacrificed to make sure the product looks good and 
ships on time. 

As we move toward the new world of merging 
traditional simulation and games we should apply 
lessons learned in both areas. 

4.3 Typical HBR Components 

One thing many different HBR approaches have in 
common is a low-fidelity description of what Al 
“means.” Figure 3 depicts the generic Al 
representation, or “Al loop,” that shows the cycle from 
what is perceived in the world, to a decision engine 
that determines what action to take, and the resulting 
action performance (which in turn modifies the world 
and leads to new perception). The components 
shown are always accounted for somewhere in an 
HBR system whether it’s a rich cognitive architecture 
or a narrowly-focused movement algorithm. 


Decision Engine 



World 


Figure 3: Generic Al Representation 

In reality the contents of the “Decision Engine” box 
are the most interesting - and the most divergent 
depending on the HBR type. Figure 4 shows a 
decomposition of different areas that have been 
modeled over the past two decades. Often, an HBR 
will attempt to represent multiple areas, e.g. memory, 
cognition, and learning. Other HBRs concentrate on 
a single area (like path finding or action selection). 


HBR types not only differ in scope and fidelity of 
human behavior emulation, they also differ in how 
they choose to represent the behavioral areas. 
Figure 5 shows some of the documented strategies 
for representation. Each representation has its 
advocates - from those who argue it better 
represents how a human brain works to those who 
argue it is the most efficient strategy. 


4.4 Architectural Building Blocks 

The state of the art in HBR could be described as 
fractious - with many competing strategies and 
definitions of the basic HBR components. Giordano 
(2004) describes the challenges of obtaining realistic 
HBR within the current state of the practice. He 
notes that some areas of HBR are farther along than 
others, including some limited conversational ability 
using state of the art speech recognition. Despite 
many advances, and many novel approaches to HBR 
from both traditional simulation and gaming 
communities, many areas are still immature, or too 
costly in terms of time and resources. 

Rather than attempting to resolve the HBR 
conundrum with a one-size-fits-all solution, we should 
instead be looking to create an open, component- 
based architecture that allows different HBR players 
to focus on specific behavioral areas. These areas 
would become building blocks to a larger HBR 
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Figure 4: Al Representation Decomposition 



Figure 5: Heterogeneous HBR Representations 
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system. Some solutions might encompass multiple 
blocks. For example a neural network may 
encompass memory and cognition. But developers 
would still need to create an API to connect to the 
rest of the system. 

Using an open architecture to deliver HBR for modern 
NPCs has other benefits as well. Just as developers 
should be providing just enough simulation to suit 
requirements, so should HBR providers seek to 
deliver just enough behavior. For some applications, 
or for some subset of the NPCs within an application, 
path-finding along with some rudimentary motivation 
may be enough to satisfy the goals. For example in a 
training application, crowds may be implemented in 
this way, while the key players (adversaries, allies) 
require a more complex brain and set of behaviors. 

A valuable lesson to be learned from the game 
industry is that processes must be streamlined to 
accommodate tight development schedules. One 
way to accomplish this is by adopting a “just enough” 
approach to HBR using an open architecture. But to 
truly trim unnecessary cost and complexity from the 
process, we should also consider unifying the NPC 
brain authoring process. Not only would HBRs need 
to show they can integrate an agent into the 
architecture, they would also need to incorporate their 
authoring tools into a single development 
environment. Advancing a single development 
environment would not be a popular constraint - as it 
requires extra effort on the part of HBR creators. But 
to meet the goals set out by the JSIMS AoA and 
Panel on Modeling Human Behavior and Command 
Decision Making seemingly radical approaches may 
be necessary. 
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